This document is dedicated to conducting the confirmatory performance analyses that were proposed for Experiment 3.
Design. The design and analysis is a 2 (difficulty: harder than reference vs. easier than reference) X 2 (difference: moderate vs. extreme) X 2 (transition: repeat vs. switch) within-subjects ANOVA on RTs and error rates.
In retrospect, I’m realizing this design doesn’t make sense because, unlike in Experiment 1, decks aren’t associated with a single level of intensity, but rather the risky deck can always have two outcomes. I think trying to coerce this analysis into the difference X difficulty design would introduce a lot of unnecessary complexity.
I’m taking a step back and thinking about what the actual insights are that I want to get out of this performance data. Are RTs and error rates slower and higher for runs of trials with more switches rather than fewer? Is the switch cost greater when switching occurs less frequently? Thinking about it this way compels me to analyze performance based on numbers of switches in a run, regardless of what deck the run comes from. It might also be interesting to explore how the actual decks influence performance, but that would be more of an exploratory performance analysis.
The analysis I’ll conduct to investigate the above is a regression where number of switches in a run and transition type predict RTs (and errors). The prediction is that RTs will be positively associated with number of switches in a run, RTs will be positively predicted by transition such that switch trials have longer RTs (i.e., the switch cost), and there will be an interaction such that the impact of transition on RT will be greater in runs where switching occurs less frequently.
Below is the cleaned data:
d <- read.csv('../../../data/dstClean.csv')
d <- d %>%
filter(transition != 'startBlock')
N <- d %>%
group_by(subject) %>%
summarize(n()) %>%
nrow(.)
head(d)
The sample size is 236.
subjectCellMeans <- d %>%
filter(transition != 'startBlock') %>%
group_by(subject, outcomeSwitch, transition) %>%
summarize(rtime = mean(cuedRt))
## `summarise()` has grouped output by 'subject', 'outcomeSwitch'. You can
## override using the `.groups` argument.
subjectCellMeans %>%
group_by(outcomeSwitch, transition) %>%
summarize(rt = mean(rtime), se = sd(rtime) / sqrt(N)) %>%
ggplot(aes(x = factor(outcomeSwitch), y = rt, group = transition)) +
geom_point(size = 2) +
geom_line(aes(linetype = transition)) +
geom_errorbar(aes(ymin = rt - se, ymax = rt + se), width = 0.5) +
labs(
x = 'Number of Switches in a Run',
y = 'Response Time (ms)'
) +
scale_linetype_discrete(name = 'Transition Type', labels = c('Repeat', 'Switch')) +
theme_bw() +
theme(legend.position = c(.9,.5),
panel.grid = element_blank())
## `summarise()` has grouped output by 'outcomeSwitch'. You can override using the
## `.groups` argument.
d$outcomeSwitchC <- d$outcomeSwitch - 8
d$transitionE <- ifelse(d$transition == 'repeat', -0.5, 0.5)
m1 <- lm(cuedRt ~ transitionE * outcomeSwitchC, data = d)
plot(m1)
summary(m1)
##
## Call:
## lm(formula = cuedRt ~ transitionE * outcomeSwitchC, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1148.0 -297.5 -134.2 127.3 4732.2
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1010.1263 1.5153 666.622 <2e-16 ***
## transitionE 299.2344 3.0306 98.738 <2e-16 ***
## outcomeSwitchC 5.1673 0.4738 10.906 <2e-16 ***
## transitionE:outcomeSwitchC -8.8521 0.9476 -9.341 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 510.5 on 135264 degrees of freedom
## Multiple R-squared: 0.08695, Adjusted R-squared: 0.08693
## F-statistic: 4294 on 3 and 135264 DF, p-value: < 2.2e-16
confint(m1)
## 2.5 % 97.5 %
## (Intercept) 1007.156347 1013.096233
## transitionE 293.294498 305.174270
## outcomeSwitchC 4.238601 6.095937
## transitionE:outcomeSwitchC -10.709466 -6.994795
newdata <- data.frame(outcomeSwitch = rep(unique(d$outcomeSwitch), 2), transition = c(rep(-0.5, length(unique(d$outcomeSwitch))), rep(0.5, length(unique(d$outcomeSwitch)))))
newdata$proba <- m1$coefficients[1] + newdata$outcomeSwitch * m1$coefficients[3] + newdata$transition * m1$coefficients[2] + newdata$transition * newdata$outcomeSwitch * m1$coefficients[4]
newdata$transitionE <- newdata$transition
newdata$outcomeSwitchC <- newdata$outcomeSwitch - 8
newdata$transition <- ifelse(newdata$transition == -0.5, 'repeat', 'switch')
newdata <- cbind(newdata, predict(m1, newdata, interval = 'predict'))
ggplot(newdata, aes(x = factor(outcomeSwitch), y = fit, group = transition)) +
geom_jitter(data = subjectCellMeans, aes(x = factor(outcomeSwitch), y = rtime, color = factor(transition)), alpha = .2, width = .2, height = 0, show.legend = FALSE) +
geom_line(size = 2, aes(color = factor(transition))) +
#geom_ribbon(aes(ymin = lwr, ymax = upr, fill = factor(transition)), alpha = .1, show.legend = FALSE) +
scale_color_manual(name = 'Transition Type', values = c(`repeat` = 'blue', `switch` = 'red'), labels = c('Repeat', 'Switch')) +
#scale_fill_manual(values = c(`repeat` = 'blue', `switch` = 'red')) +
labs(
x = 'Number of Switches in a Run',
y = 'Response Time (ms)',
caption = 'Lines reflect predictions from regression. Points reflect subject-wise cell means.'
) +
theme_bw() +
theme(legend.position = c(.8,.85),
panel.grid = element_blank())
Centering at high number of switches per run (12)
d$outcomeSwitchC <- d$outcomeSwitch - 12
d$transitionE <- ifelse(d$transition == 'repeat', -0.5, 0.5)
m1 <- lm(cuedRt ~ transitionE * outcomeSwitchC, data = d)
summary(m1)
##
## Call:
## lm(formula = cuedRt ~ transitionE * outcomeSwitchC, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1148.0 -297.5 -134.2 127.3 4732.2
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1030.7954 2.3838 432.421 <2e-16 ***
## transitionE 263.8259 4.7676 55.338 <2e-16 ***
## outcomeSwitchC 5.1673 0.4738 10.906 <2e-16 ***
## transitionE:outcomeSwitchC -8.8521 0.9476 -9.341 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 510.5 on 135264 degrees of freedom
## Multiple R-squared: 0.08695, Adjusted R-squared: 0.08693
## F-statistic: 4294 on 3 and 135264 DF, p-value: < 2.2e-16
confint(m1)
## 2.5 % 97.5 %
## (Intercept) 1026.123207 1035.467527
## transitionE 254.481542 273.170182
## outcomeSwitchC 4.238601 6.095937
## transitionE:outcomeSwitchC -10.709466 -6.994795
And centering at low number of switches per run (4)
d$outcomeSwitchC <- d$outcomeSwitch - 4
d$transitionE <- ifelse(d$transition == 'repeat', -0.5, 0.5)
m1 <- lm(cuedRt ~ transitionE * outcomeSwitchC, data = d)
summary(m1)
##
## Call:
## lm(formula = cuedRt ~ transitionE * outcomeSwitchC, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1148.0 -297.5 -134.2 127.3 4732.2
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 989.4572 2.4686 400.822 <2e-16 ***
## transitionE 334.6429 4.9371 67.781 <2e-16 ***
## outcomeSwitchC 5.1673 0.4738 10.906 <2e-16 ***
## transitionE:outcomeSwitchC -8.8521 0.9476 -9.341 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 510.5 on 135264 degrees of freedom
## Multiple R-squared: 0.08695, Adjusted R-squared: 0.08693
## F-statistic: 4294 on 3 and 135264 DF, p-value: < 2.2e-16
confint(m1)
## 2.5 % 97.5 %
## (Intercept) 984.618866 994.295562
## transitionE 324.966211 344.319602
## outcomeSwitchC 4.238601 6.095937
## transitionE:outcomeSwitchC -10.709466 -6.994795
Looking at the relationship between switches in a run and mean RT (dropping transition)
m2 <- lm(cuedRt ~ outcomeSwitch, data = d)
slope <- round(m2$coefficients[2], 2)
summary(m2)
##
## Call:
## lm(formula = cuedRt ~ outcomeSwitch, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1072.4 -329.2 -142.1 153.6 4983.6
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 809.0125 3.8598 209.60 <2e-16 ***
## outcomeSwitch 24.1021 0.4484 53.76 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 528.6 on 135266 degrees of freedom
## Multiple R-squared: 0.02092, Adjusted R-squared: 0.02091
## F-statistic: 2890 on 1 and 135266 DF, p-value: < 2.2e-16
confint(m2)
## 2.5 % 97.5 %
## (Intercept) 801.44741 816.57759
## outcomeSwitch 23.22335 24.98094
sMeans <- d %>%
group_by(subject, outcomeSwitch) %>%
summarize(rtime = mean(cuedRt))
## `summarise()` has grouped output by 'subject'. You can override using the
## `.groups` argument.
sMeans %>%
group_by(outcomeSwitch) %>%
summarize(rt = mean(rtime), se = sd(rtime) / sqrt(N)) %>%
ggplot(aes(x = factor(outcomeSwitch), y = rt, group = 1)) +
geom_line() +
geom_point(size = 2.5) +
geom_jitter(data = sMeans, aes(x = factor(outcomeSwitch), y = rtime), alpha = .2, width = .1, height = 0) +
geom_errorbar(aes(ymin = rt - se, ymax = rt + se), width = 0.5) +
annotate('text', x = 4, y = 1500, label = paste('b = ', slope, sep = '')) +
labs(
x = 'Number of Switches in a Run',
y = 'Response Time (ms)'
) +
theme_bw() +
theme(panel.grid = element_blank())
Below is the cleaned data:
d <- read.csv('../../../data/dstCleanErrors.csv')
d <- d %>%
filter(d$transition != 'startBlock')
N <- d %>%
group_by(subject) %>%
summarize(n()) %>%
nrow(.)
d
The sample size is 236.
subjectCellMeans <- d %>%
group_by(subject, outcomeSwitch, transition) %>%
summarize(error = mean(error))
## `summarise()` has grouped output by 'subject', 'outcomeSwitch'. You can
## override using the `.groups` argument.
subjectCellMeans %>%
group_by(outcomeSwitch, transition) %>%
summarize(err = mean(error), se = sd(error) / sqrt(N)) %>%
ggplot(aes(x = factor(outcomeSwitch), y = err, group = transition)) +
geom_point(size = 2) +
geom_line(aes(linetype = transition)) +
geom_errorbar(aes(ymin = err - se, ymax = err + se), width = 0.5) +
labs(
x = 'Number of Switches in a Run',
y = 'Error Rate'
) +
scale_linetype_discrete(name = 'Transition Type', labels = c('Repeat', 'Switch')) +
theme_bw() +
theme(legend.position = c(.2,.8),
panel.grid = element_blank())
## `summarise()` has grouped output by 'outcomeSwitch'. You can override using the
## `.groups` argument.
d$outcomeSwitchC <- d$outcomeSwitch - 8
d$transitionE <- ifelse(d$transition == 'repeat', -0.5, 0.5)
m1 <- lm(error ~ transitionE * outcomeSwitchC, data = d)
plot(m1)
summary(m1)
##
## Call:
## lm(formula = error ~ transitionE * outcomeSwitchC, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.04935 -0.04838 -0.03101 -0.02826 0.97358
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.0385388 0.0005403 71.324 <2e-16 ***
## transitionE 0.0187210 0.0010807 17.324 <2e-16 ***
## outcomeSwitchC 0.0003508 0.0001694 2.071 0.0384 *
## transitionE:outcomeSwitchC -0.0002167 0.0003389 -0.639 0.5226
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.192 on 150432 degrees of freedom
## Multiple R-squared: 0.002631, Adjusted R-squared: 0.002612
## F-statistic: 132.3 on 3 and 150432 DF, p-value: < 2.2e-16
confint(m1)
## 2.5 % 97.5 %
## (Intercept) 3.747979e-02 0.0395978911
## transitionE 1.660295e-02 0.0208391426
## outcomeSwitchC 1.874053e-05 0.0006829436
## transitionE:outcomeSwitchC -8.808535e-04 0.0004475527
newdata <- data.frame(outcomeSwitch = rep(unique(d$outcomeSwitch), 2), transition = c(rep(-0.5, length(unique(d$outcomeSwitch))), rep(0.5, length(unique(d$outcomeSwitch)))))
newdata$proba <- m1$coefficients[1] + newdata$outcomeSwitch * m1$coefficients[3] + newdata$transition * m1$coefficients[2] + newdata$transition * newdata$outcomeSwitch * m1$coefficients[4]
newdata$transitionE <- newdata$transition
newdata$outcomeSwitchC <- newdata$outcomeSwitch - 8
newdata$transition <- ifelse(newdata$transition == -0.5, 'repeat', 'switch')
newdata <- cbind(newdata, predict(m1, newdata, interval = 'predict'))
ggplot(newdata, aes(x = factor(outcomeSwitch), y = fit, group = transition)) +
geom_jitter(data = subjectCellMeans, aes(x = factor(outcomeSwitch), y = error, color = factor(transition)), alpha = .2, width = .2, height = 0, show.legend = FALSE) +
geom_line(size = 2, aes(color = factor(transition))) +
#geom_ribbon(aes(ymin = lwr, ymax = upr, fill = factor(transition)), alpha = .1, show.legend = FALSE) +
scale_color_manual(name = 'Transition Type', values = c(`repeat` = 'blue', `switch` = 'red'), labels = c('Repeat', 'Switch')) +
#scale_fill_manual(values = c(`repeat` = 'blue', `switch` = 'red')) +
labs(
x = 'Number of Switches in a Run',
y = 'Error Rate',
caption = 'Lines reflect predictions from regression. Points reflect subject-wise cell means.'
) +
ylim(0,.2) +
theme_bw() +
theme(legend.position = c(.8,.85),
panel.grid = element_blank())
## Warning: Removed 79 rows containing missing values (geom_point).
Looking at the relationship between switches in a run and mean RT (dropping transition)
m2 <- lm(error ~ outcomeSwitch, data = d)
slope <- round(m2$coefficients[2], 5)
summary(m2)
##
## Call:
## lm(formula = error ~ outcomeSwitch, data = d)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.04754 -0.04144 -0.03839 -0.03381 0.97076
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.0261839 0.0013432 19.493 <2e-16 ***
## outcomeSwitch 0.0015258 0.0001552 9.829 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.1922 on 150434 degrees of freedom
## Multiple R-squared: 0.0006418, Adjusted R-squared: 0.0006351
## F-statistic: 96.61 on 1 and 150434 DF, p-value: < 2.2e-16
confint(m2)
## 2.5 % 97.5 %
## (Intercept) 0.023551134 0.028816608
## outcomeSwitch 0.001221531 0.001830049
sMeans <- d %>%
group_by(subject, outcomeSwitch) %>%
summarize(error = mean(error))
## `summarise()` has grouped output by 'subject'. You can override using the
## `.groups` argument.
sMeans %>%
group_by(outcomeSwitch) %>%
summarize(err = mean(error), se = sd(error) / sqrt(N)) %>%
ggplot(aes(x = factor(outcomeSwitch), y = err, group = 1)) +
geom_line() +
geom_point(size = 2.5) +
geom_jitter(data = sMeans, aes(x = factor(outcomeSwitch), y = error), alpha = .2, width = .1, height = 0) +
geom_errorbar(aes(ymin = err - se, ymax = err + se), width = 0.5) +
annotate('text', x = 4, y = .3, label = paste('b = ', slope, sep = '')) +
labs(
x = 'Number of Switches in a Run',
y = 'Error Rate'
) +
theme_bw() +
theme(panel.grid = element_blank())
A work by Dave Braun
dab414@lehigh.edu